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Engaging First-year University Students in Research: Promise, Potentials, 
and Pitfalls 


Abstract 

In 2014, the Undergraduate Research Initiative at the University of Saskatchewan implemented a pilot project 
to organize, support, and promote curriculum-based research experience as an integral aspect of participating 
first-year courses. The framework for the course-based initiative was the research arc; usually in groups, 
students in these classes would develop a research question, investigate it using discipline-appropriate 
methodologies, and disseminate the results. Nine classes (Agriculture, Animal Bioscience, Environmental 
Science, Womens and Gender Studies, Psychology, Kinesiology, and Interdisciplinary Studies) participated in 
this program pilot. There were four key agents in the program: faculty instructors, research coaches, students 
in participating first-year classes, and university administrative staff. 

This preliminary evaluation of the pilot suggests that first-year undergraduate research experiences have 
potential to benefit the undergraduate student participants as well as the faculty and research coaches 
involved. The primary benefits that faculty reported experiencing included an increased interest in ways to 
engage learners, reexamination of and reflection on their teaching strategies, the pragmatic support of a 
research coach helping with their work load, and an invigoration of their research. The primary benefits to 
research coaches included enhancement of their professional skills, experience in lesson planning and 
facilitation, CV building, and an ideology shift in how to best facilitate learning for undergraduate students. 
The most prominent benefits for undergraduate students appeared to be that they gained a better idea about 
how researchers think and work, that they increased their understanding of how research works, and that their 
own research and professional skills had improved. 

Early, bottom-up evaluation identified characteristics of implementation that appear to best facilitate 
achievement of the initiative s outcomes and identified the potential pitfall of imposing outcomes, from 
related but distinct initiatives, that may not be achievable or optimal in in the setting of first-year classes. The 
results of this evaluation suggest that rather than gaining clarity or focus, first-year students in course-based 
research experiences might gain awareness of their personal potential, of the potential of research, and of their 
career/educational options. 

En 2014, l’Universite de la Saskatchewan a mis en oeuvre un projet pilote, la Undergraduate Research 
Initiative (initiative de recherche pour etudiants de premier cycle) dont l’objectif etait d’organiser, de soutenir 
et de promouvoir l’experience de recherche basee sur le programme de cours et faisant directement partie des 
cours de premiere annee participants. Le cadre de cette initiative basee sur les cours etait l’arc de recherche; 
habituellement divises en groupes, les etudiants inscrits dans ces cours ont elabore une question de recherche, 
Font examinee par le biais de methodologies appropriees a la discipline et en ont diffuse les resultats. Neuf 
cours (agriculture, sciences biologiques animates, sciences de l’environnement, etudes sur les femmes et le 
genre, psychologie, kinesiologie et etudes interdisciplinaires) ont participe a ce programme pilote. 11 y avait 
quatre agents cles dans le programme : les professeurs, les accompagnateurs de recherche, les etudiants inscrits 
dans les cours de premiere annee participants et le personnel administratif de l’universite. 

Cette evaluation preliminaire du projet pilote suggere que les experiences de recherche en premiere annee 
d’un programme de premier cycle peuvent potentiellement offrir des avantages aux etudiants de premier cycle 
qui y participent ainsi qu’aux professeurs et aux accompagnateurs de recherche. Les avantages principaux, 
selon les professeurs qui ont fait un rapport sur leur experience, comprennent un plus grand interet dans les 
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manieres d’engager les apprenants, le reexamen dune reflexion sur leurs strategies d’enseignement, le soutien 
pragmatique des accompagnateurs de recherche qui les aident avec leur charge de travail et une revitalisation 
de leur recherche. Les avantages principaux pour les accompagnateurs de recherche comprennent une 
amelioration de leurs competences professionnelles, une experience dans la planification et la facilitation des 
lecons, des experiences a ajouter a leur curriculum vitae et un changement ideologique concernant la 
meilleure maniere de faciliter l’apprentissage des etudiants de premier cycle. Les avantages les plus importants 
pour les etudiants de premier cycle semblent etre qu’ils ont acquis une meilleure comprehension de la 
maniere dont les chercheurs pensent et travaillent et de la maniere dont la recherche fonctionne, et que leurs 
propres recherches et leurs propres competences professionnelles ont ete ameliorees. 

Levaluation preliminaire participative a identifie des caracteristiques de mise en oeuvre qui semblent faciliter 
au mieuxl’atteinte des resultats de 1’initiative et a identifie Feared potentiel d’imposer des resultats, apartir 
d’initiatives differentes mais connexes, qui risquent de ne pas etre realisables ou optimales dans le cadre de 
cours de premiere annee. Les resultats de cette evaluation suggerent que, plutot que de gagner en clarte ou en 
focus, les etudiants de premiere annee qui participent a des experiences de recherche basees sur les cours 
pourraient prendre conscience de leur potentiel personnel, du potentiel de la recherche et de leurs options de 
carriere ou scolaires. 
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The Undergraduate Research (UGR) Initiative is a joint venture between the offices of 
the Vice-Provost Teaching and Learning and the Vice-President Research at the University of 
Saskatchewan. The initiative’s principal goal is to ensure every undergraduate student graduating 
from the University of Saskatchewan engages in a meaningful research, scholarly, or artistic 
experience. The initiative is supporting a culture of research at the university undergraduate level 
by expanding opportunities across disciplines and academic years, increasing visibility, and 
improving recognition. The UGR initiative co-funds, supports, and promotes undergraduate 
research experiences (UREs) which include opportunities such as summer research assistantships 
and course-based research as well as mechanisms for knowledge mobilization such as an 
undergraduate research journal. 

UREs are associated with a variety of positive outcomes. Student participants typically 
experience personal and professional development, begin to think and work like researchers, 
hone academic skills, clarify and refine their career and educational pathways, and become better 
prepared for graduate school (Hunter, Laursen, & Seymour, 2007; Lopatto, 2009; Seymour, 
Hunter, Larsen, & Deantoni, 2004). The most common and familiar UREs typically follow the 
apprenticeship model, and involve students competing for placements, spending time in a 
research laboratory, and receiving one-on-one mentoring from a postdoc, graduate student, or 
faculty member (Linn, Palmer, Baranger, Gerard, & Stone, 2015). A newer and somewhat lesser- 
known type of URE is the course-based research experience (CURE), which provides research 
experiences for a class of students guided by an instructor often with the help of graduate 
students, and involve classes, credits, grades, and assignments (Linn et al., 2015). 

The outcomes of CUREs are less established than those of the typical apprenticeship 
model UREs. Some studies of CUREs report student gains in research skills, self-efficacy, and 
intent to persist in their field (Corwin Auchincloss et al., 2014; Harrison, Dunbar, Ratmansky, 
Boyd, & Lopatto,, 2011; Jordan et al., 2014; Lopatto et al., 2008; Rowland, Lawrie, Behrendorff, 
& Gillam, 2012; Shaffer et al., 2010). CUREs have most frequently been implemented in science 
courses. The majority of the few published evaluations of CUREs (i.e., Corwin Auchincloss et 
al., 2014; Harrison et al., 2011; Jordan et al., 2014; Lopatto et al., 2008; Rowland et al., 2012; 
Shaffer et al., 2010) have been offshoots of the American CUREnet initiative funded by the 
National Science Foundation with the explicit purpose of “helping students understand core 
concepts in biology, develop core scientific competencies, and become active, contributing 
members of the scientific community” (CUREnet, 2015). Therefore, these few evaluations have 
focused on the outcomes of CUREs for science students, in science classes. Relatively less is 
known about the outcomes of CUREs for humanities or social science students. The evaluation 
reported in this remainder of this article can supplement the existing evidence for CUREs by 
reporting the outcomes of CUREs for students in the humanities and social sciences in addition 
to science students. 

The CURE Pilot Project 

In 2014, the UGR initiative at the University of Saskatchewan implemented a pilot 
project to organize, support, and promote curriculum-based research experience as an integral 
aspect of participating first-year courses. The framework for the course-based initiative was the 
research arc; usually in groups, students in these classes would develop a research question, 
investigate it using discipline-appropriate methodologies, and disseminate the results. Nine 
classes (in Agriculture, Animal Bioscience, Environmental Science, Women’s and Gender 
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Studies, Psychology, Kinesiology, and Interdisciplinary Studies) participated in this program 
pilot. There were four key agents in the program: faculty instructors, research coaches, students 
in participating first-year classes, and university administrative staff. 

Faculty instructors. Faculty planned and implemented the target courses as well as 
structured the research experiences. At the outset of the pilot, potentially interested faculty were 
approached as individuals or departments once open invitations to participate were made 
available by administrators to Associate Deans of Research in targeted colleges. Collaborations 
ensued when, through a series of meetings, faculty expressed interest in offering research 
experiences which incorporated the tenants of the program (the research arc). These meetings 
involved the exchange of ideas and tailored curriculum redesign support. Faculty attended 
workshops, face-to-face events, and engaged in individual or small group consultations with 
UGR staff and other instructors in order to learn new research-based teaching and cooperative 
learning strategies, to explore ethical considerations, and to better understand and influence the 
UGR initiative’s objectives and philosophies. 

Research coaches. Graduate students and advanced undergraduate students were 
employed by the initiative as research coaches. They attended a training workshop covering an 
introduction to professional conduct, the university learning charter, the UGR initiative’s goals 
and complementary programming, as well as facilitation, teamwork, and interpersonal skills. 
Research coaches also collaborated with the course instructor and the first-year students as the 
research experience was planned, carried out, and communicated. Some research coaches 
supplemented the initial curriculum-based research project design with additional resources and 
targeted support. They coached undergraduate students in completing the project and provided 
assistance in carrying out the opportunities for students to share their projects with classmates. 
Coaches answered student questions and guided and supported students regarding research 
processes. In some cases research coaches were involved with assessing student work and 
providing feedback. 

Undergraduate students. Undergraduate students engaged in their course-based 
research experience. Under the direction of their faculty instructors and research coaches, 
students participated in the research experience, which involved contributing to a research 
question, investigating that question by collecting and/or analyzing data, and disseminating 
results. 

UGR staff. The UGR staff from both the offices of the Vice-Provost Teaching and 
Learning and the Vice-President Research facilitated the entire process. They approached, 
recruited and trained research coaches, supported participating faculty, and coordinated and 
promoted the UGR initiative on campus and beyond. 

The research experience was implemented differently in each class in order to meet 
faculty preferences and the needs of the course learning objectives. Some classes conducted in- 
class surveys, using students as the survey sample while other classes analyzed pre-existing, 
publically available data sets (e.g., climate data). Most classes were structured to have students 
work in groups, but in some instances students worked individually. Participating classes used 
the full range of qualitative, quantitative, and mixed methods in evolving the research 
component. A number of classes had researchers visit and explain their work as guest speakers, 
or went on fieldtrips to visit campus laboratories. In some cases students or groups shared their 
research findings with the whole class or their instructor(s) via reports or presentations, while 
others took part in public poster sessions to disseminate their discoveries. In some classes 
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students’ research-based work was awarded grades according to quality while in other classes 
students received credit for participating in the project. 

Theoretical Framework for the Evaluation 

At the time of this evaluation, the UGR initiative was in its pilot phase. Thus, there was 
little official direction in terms of exactly which aspects of the initiative should be evaluated; 
there was simply a consensus that it should be evaluated. In order to create a framework for the 
evaluation, the UGR staff and graduate student evaluator reviewed previous research regarding 
outcomes of CUREs (e.g., Corwin Auchincloss et al., 2014; Harrison et al., 2011; Jordan et al., 
2014; Lopatto et al., 2008; Rowland et al., 2012; Shaffer et al., 2010) and URE’s (Hunter et al., 
2007; Lopatto, 2009; Seymour et al., 2004). In addition to outcomes suggested by the research 
literature, the UGR staff identified a number of outcomes they individually expected the 
initiative would achieve. Results from the previous literature and the individual expectations of 
the UGR staff were combined into three logic models for the initiative: one focused on the 
undergraduate students (Figure 1); one focused on the research coaches (Figure 2); and one 
focused on the faculty instructors (Figure 3). 

The logic models are graphic representations of the theoretical relationships between 
inputs, activities, and results of the initiative. They represent the hypothesized causal linkages 
between the activities the initiative engages in and the changes those activities are intended to 
produce. By combining findings from the literature and expectations from the UGR staff, the 
logic models provide a theoretical framework for what the initiative is supposed to achieve and, 
thus, what is evaluated in the remainder of this paper. 

Scope and Type and Theoretical Approach of the Evaluation 

In order to inform improvement for the initiative, the evaluation was focused on process 
and initial outcomes. The evaluation was largely formative, targeted on understanding 
organization and fidelity of implementation, the strengths and weaknesses of implementation, 
how implementation could be improved, and to gauge to what extent the initiative is on the right 
path to creating intended positive social change on campus. Thus, the evaluation took a bottom- 
up approach (Chen & Garbe, 2011). That is, rather than conducting an efficacy evaluation for an 
initiative that was still evolving, the evaluation focused on viability, and preliminary perceptions 
of effectiveness. This focus can be more useful in early stages of program development as it 
allows program decision-makers to make changes before finalizing an intervention. It is 
anticipated that if evidence is found for the viability and perceived effectiveness the UGR 
initiative, more rigorous evaluation methods can be invested in, in the future. The purpose of the 
evaluation at this stage is to see if the initiative holds promise and if there are potential pitfalls to 
avoid in future implementation. With this in mind, certain preliminary outcomes (from the logic 
models below) became the focus of the evaluation. The evaluation questions included: 
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Inputs 


Activities 


Participants 


Short-term 

outcomes 


Intermediate 

outcomes 


Long-term 

outcomes 


Staff 

Volunteers 

Time 

Money 

Research 

Materials 

Equipment 

Technology 

Partners 

Faculty 


Faculty/ Instructor 
training: 

Workshops 
Face-to-face events 
Individual 
consultations 
(teaching strategies; 
cooperative 
learning; UGR 
resources, strategies, 
and goals) 


Research coach 
training: 

Workshops 
Information sessions 
(professional 
conduct; program 
goals; facilitation; 

teamwork; 
interpersonal skills) 


First-year, in 
class, curriculum 
based, 

undergraduate 

research 

experience 



Figure 1. Logic model of student outcomes. 


http://ir.lib.uwo.ca/cj so tl_rcacea/vol 7 /issi /3 


4 




























Sangster et al.: Promise, Potentials, and Pitfalls 


Inputs 


Activities 


Participants 


Short-term 

outcomes 


Intermediate 

outcomes 


Long-term 

outcomes 


Staff 

Volunteers 

Time 

Money 

Research 

Materials 

Equipment 

Technology 

Partners 

Faculty 


Faculty/ Instructor 
training: 

Workshops 
Face-to-face events 
Individual 
consultations 
(teaching strategies; 
cooperative 
learning; UGR 
resources, strategies, 
and goals) 


Research coach 
training: 

Workshops 
Information sessions 
(professional 
conduct; program 
goals; facilitation; 

teamwork; 
interpersonal skills) 


First-year, in 
class, curriculum 
based, 

undergraduate 

research 

experience 


Research coaches 

(Upper-level UG 
and graduate 
students) 


Increased awareness of 
the university as a 
community of 
researchers and 
learners at all levels 


Enhancement of 
University research 
culture 


Improvement in 
research & 
professional skills 


Engagement in 
scholarship of teaching 
and learning 


Increased awareness of 
alternative pedagogical 
practices 


Increased awareness of 
the connection 
between research and 
teaching 


> 


Research coaches are 
more academically 
successful 


Increased 
integration of 
research and 
teaching in future 
practice 


Increased 
application of 
alternative 
pedagogical 
practices in future 
practice 


Research coaches have 
increased 
employability in 
academic and related 
fields 


The University has an 
improved reputation 
(research/teaching/ 
student engagement) 


Students (non- 
traditional students, in 
particular) are more 
academically 
successful 


Figure 2. Logic model of research coach outcomes. 
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outcomes 

■ 
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■ 
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Time 
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Research 
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Technology 

Partners 
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Faculty/ Instructor 
training: 

Workshops 
Face-to-face events 
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consultations 
(teaching strategies; 
cooperative 
learning; UGR 
resources, strategies, 
and goals) 


Research coach 
training: 

Workshops 
Information sessions 
(professional 
conduct; program 
goals; facilitation; 

teamwork; 
interpersonal skills) 


First-year, in 
class, curriculum 
based, 

undergraduate 

research 

experience 


Increased awareness of 
alternative pedagogical 
practices 


Increased application 
of alternative 
pedagogical practices 
outside the UGR 
initiative 




Reinvigoration in 


Increased faculty 

> 

research 

* 

research productivity 


Students (non- 
traditional students, in 
particular) are more 
academically 
successful 


i 


The University has an 
improved reputation 
(research/teaching/ 
student experience) 


Enhancement of 
University research 
culture 


Increase in range of 


Increase in value 

research conducted by 

—► 

granted to range of 

faculty 


research types 


Figure 3. Logic model of faculty/instructor outcomes. 
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Students 

1. Do students have increased academic clarity? 

• clarity about which courses to take in the future 

• clarity about which field of study in which to major 

• clarity about their career interests 

• clarity about their interest in graduate studies 

2. Do students have increased awareness of the course’s discipline and its research process? 

3. Have students’ professional skills improved? 

4. Are students more engaged in learning? 

5. Are students beginning to identify as student researchers? 

• Do they have in increased understanding of how researchers think and work? 

• Are they beginning to perceive themselves as researchers? 

• Are they beginning to see they have a role in research on campus? 

Research Coaches 

6. Are research coaches benefitting from their experience? 

• Are their professional skills improving? 

F acuity/instructors 

7. Are faculty/instructors benefitting from their experience? 

• Are they learning new teaching methods? 

8. Do they see teaching and research as more connected? 

9. Are they more invigorated about research? 

Program Implementation 

10. Are there aspects of implementation that seem to best facilitate achievement of the 
initiative’s expected outcomes? 

The evaluation was conducted using an “inclusive evaluation” (Mertens, 1999) approach 
and involved three primary participant groups - the faculty instructors, the research coaches, and 
the students - occupying a range of power positions on the University of Saskatchewan campus. 
The inclusive evaluation approach focused on supporting each participant group being heard 
rather than emphasizing one voice at the expense of another. As well as promoting equity on 
campus, this approach enhanced the quality and relevance of the evaluation by attempting to 
capture all the important perspectives on the implementation and outcomes of the initiative, 
rather than overstating the views of those in possession of more influence and power. 

Method 

The evaluation involved a combination of qualitative and quantitative methods to 
accommodate the aforementioned theoretical approach. Surveys, interviews, and focus groups 
were conducted in order to triangulate data sources with regards to issues of implementation and 
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outcome achievement. Utilizing a variety of quantitative and qualitative measures enhances the 
extent to which we can be confident in the evaluation’s conclusions and recommendations. All 
data collection occurred under the considerations of ethical research conduct as outlined in the 
Government of Canada’s Tri-Council Ethical Guidelines, however, the project was considered 
strictly program evaluation, and thus exempt from ethics review by the University of 
Saskatchewan Behavioral Research Ethics Board. 

Surveys 

Self-administered, in-person surveys were conducted to evaluate implementation, 
organization, and outcomes of the pilot as outlined in the logic models. Three surveys were 
conducted: (1) surveys of faculty (N= 10; 100%); (2) surveys of research coaches (N = 14; 
100%); and (3) surveys of students (N = 622; 58%). Surveys were kept at minimal length to 
encourage participation. Student surveys were conducted in-class, using OpScan or “bubble” 
response sheets, excluding two classes for which the instructors were unable to provide class¬ 
time for the survey and chose to have an online survey available for their students to fill out in 
their own time. The OpScan sheets allowed for quick responses and facilitated data entry and 
summary. The in-class delivery led to higher response rates than the online survey as very few 
students from the classes with the online survey option completed it (zero from one class of 28 
and only six of 72 students from the other). The natural and social sciences and humanities were 
fairly well represented in the survey responses from the students (Agriculture n = 151; Animal 
bioscience n = 81; Environmental science n = 20; Interdisciplinary studies n = 11; Kinesiology n 
= 101; Psychology n = 200; Women’s and gender studies n = 58). The student survey (see 
appendix A) consisted of 23 items assessing students’ perception of the research experience’s 
implementation (e.g., “I thought the research experience was adequately organized”) and 
potential benefits to them as participants (e.g., “I think I have an increased understanding of how 
research works because of the research experience”). Response options depicting a five-point 
scale from “Strongly Disagree” to “Strongly Agree” were available. 

Faculty instructor and research coach surveys were in-person, pen and paper surveys and 
conducted at the outset of the interviews and focus groups with these two groups. The research 
coach survey (see appendix B) consisted of 26 items assessing participants’ perception of the 
program’s implementation (e.g., “Overall, I was satisfied with how the UGR experience was 
implemented in class”); potential benefits for themselves (e.g., “The research coach experience 
helped me develop my professional skills, including critical thinking, problem solving, and 
interpersonal skills); and their perception of benefits experienced by the participating 
undergraduate students who they supported or instructed (e.g., “I believe the undergraduate 
students are more engaged in learning as a result of the in-class research experience”). The 
instructor survey (see appendix C) consisted of 32 items that were designed to evaluate faculty 
perception of implementation and potential benefits for them (e.g., “I feel more invigorated about 
research as a result of the experience), for the research coaches with whom they worked (e.g., I 
believe the research coaches will have increased employability in academic or related fields as a 
result of participating as a research coach”), and for the undergraduate students they were 
teaching (e.g., “I think the undergraduate students began to understand they have a role in 
research on campus”). The instructor and research coach surveys also had response options on a 
five-point scale ranging from “Strongly Disagree” to “Strongly Agree”. The surveys were 
developed by the graduate student evaluator in conjunction with representatives from the offices 
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of the Vice-Provost Teaching and Learning and the Vice-President Research and adhered to best 
practices with regards to survey design (see Dillman, Smyth, & Melani, 2011). 

The three surveys were designed to triangulate the perceptions of each of the participant 
groups as much as possible to obtain a balance of views on the evaluation questions. For 
instance, from the evaluation question listed above of “Do students have increased academic 
clarity (about their career interests)?” emerged the survey item for the undergraduates: “I have a 
better idea about what I would like to pursue (or not pursue) as a career because of the research 
experience” and one survey item for research coaches and faculty instructors: “I think the 
research experience provided the undergraduate students increased clarity regarding their career 
interests.” Each group was also asked to respond to additional items that were only relevant to 
their group. 

Interviews and Focus Groups 

Interviews and focus groups were conducted to evaluate implementation, organization, 
and preliminary outcomes of the initiative. Faculty (N = 10; 100%) and research coaches (N = 
14; 100%) participated in interviews and focus groups. Some participated in interviews while 
others engaged in focus groups, and this was based on scheduling, availability, and the extent to 
which individuals may have had common experiences. Interviews and focus groups were 
approximately 45 minutes to an hour, face-to-face, and semi-structured. Interviews were 
recorded and transcribed verbatim with transcripts thematically analyzed (Braun & Clarke, 2006) 
to target responses that provided insight into the evaluation questions. After becoming familiar 
with the transcripts through several pre-reads, the phase of generating initial codes began. In this 
stage, features of the data that were meaningful in regard to evaluation questions were noted 
through line-by-line coding. Once all codes were generated, they were organized into initial 
themes and subthemes and the way they addressed the evaluation questions was explored. The 
next stage involved reviewing themes and their associated codes to ensure that each theme aided 
in telling a story about the data in relation to the evaluation questions. In making decision rules 
as to what “counted” as a theme, prevalence was considered, as well as whether it captured 
something important in relation to the evaluation questions (Braun & Clark, 2006). 

Interviews and focus groups with the undergraduate students were also included in the initial 
evaluation plan. Unfortunately, no students responded to the invitation to participate. The 
interviews and focus groups were scheduled during an exam period, and this may have 
contributed to low response rates. 

The interview/focus group guides for the research coaches (see appendix D) and faculty 
instructors (see appendix E) were designed to triangulate with and supplement the survey 
responses. For instance, in the surveys, research coaches and instructors were asked the extent to 
which they agreed that research coaches benefitted by participating in the initiative, while in the 
interviews and focus groups they were asked in what ways they thought that participating in the 
initiative might contribute to their success. The surveys focused on perceptions of if 
implementation went according to plan and if they perceived outcomes to be achieved. The 
interviews and focus groups focused on how and why outcomes occurred and how 
implementation could be improved. The interviews and focus groups allowed for spontaneous 
and unanticipated responses, while the surveys included only closed-ended items. 
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Results 


Undergraduate Students 

Academic clarity. It was expected that students would develop increased clarity about 
what courses they may have become interested in taking subsequently, and also what resulting 
discipline they may have become interested in pursuing. The theory of change underlying these 
outcomes is that students would get a better understanding of a discipline from the research 
experience, and get a better idea of their level of interest in courses with a substantial research 
component. Further, it was postulated that students might get a better idea of their potential 
interest in pursuing graduate studies or what career they may have become interested in. On 
average students’ survey responses suggested they were unwilling to agree that as a result of the 
course-based research experience they had a better idea of what courses they would like to take 
in the future, what they would like to major in, what field they would like to pursue, or if they 
were interested in graduate studies. In their survey responses instructors and research coaches 
were similarly skeptical about correlating the research experience to students’ clarity about these 
issues. Instructors were more consistent in their responses than research coaches whose 
responses greatly varied (see Figures 4, 5, 6, and 7). 



Strongly Disagree Neither Agree Strongly 
disagree agree nor agree 

disagree 


□ Students (M = 3.07) 

□ Research Coaches (M = 3.07) 

□ Instructors (M = 3.30) 


Figure 4. Increased clarity about future courses. 
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Strongly Disagree Neither Agree Strongly 
disagree agree nor agree 

disagree 


□ Students (M = 2.91) 

□ Research Coaches(M = 3.27) 

□ Instructors (M = 3.30) 


re 5. Increased clarity about major. 



Strongly Disagree Neither Agree Strongly 
disagree agree nor agree 

disagree 


■ Students (M = 2.92) 

□ Research Coaches(M = 3.07) 

□ Instructors (M = 3.30) 


Figure 6. Increased clarity about career interests. 
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■ Students (M = 3.05) 

□ Research Coaches (M = 3.27) 

□ Instructors (M = 3.20) 


Figure 7. Increased clarity about interest in graduate studies. 


While in the research coach and faculty focus groups and interviews there was general 
consensus around students having gained clarity about what types of courses they might be 
interested in taking in the future, skepticism persisted regarding whether students would have 
obtained any clarity regarding graduate studies or career paths. Interestingly, there was a 
common impression that the research exposure in the first-year research experience might make 
students less hesitant to take certain courses in the future, especially lab courses or research 
methods courses. For instance, one faculty member commented: 


I think what we did was we eliminated methodology and survey phobia. Those students 
may take our methodology course because they were exposed to this. Many of our 
students would not have even thought of taking that course [before this experience]. 


In terms of career interests, faculty suggested in the interviews and focus groups that it might not 
have provided clarity as much as awareness of more career options or possibilities. Faculty 
described how, traditionally, in certain departments, the majority of first-year students have a 
one-track mind regarding their future careers. For instance, that an abundance of students in first- 
year animal bioscience typically have plans to become veterinarians or that many first-year 
kinesiology students assume they will become physiotherapists. Instructors were pleased to 
report that the research experience provided the students increased awareness of other careers 
opportunities related to the field: “it kind of broadened their horizons a bit”; “...you could see 
that there were so many interests in that class and I think projects like that allow students to 
explore career paths....” 

Finally, a few faculty expressed concern over the idea of clarity being a goal of the 
initiative and suggested that increased awareness of options and possibilities, in general, would 
be a more appropriate goal. Faculty described an undergraduate degree as an ideal time for 
students to explore their options: “the more varied the experience can be for them, the more 
different kinds of research experiences they can have, from my point of view, is better.” Faculty 
with this perspective outlined how the first year research experience can help to expose students 
to a number of ways to think about the world and how learning to consider multiple perspectives 
will be more beneficial than picking a single field (i.e., perspective) early on because the 
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workplace more than ever requires adaptability. 

In sum, students did not agree that they experienced any clarity regarding their future 
plans due to the research experience. Further, faculty advised that rather that clarity, which might 
suggest a narrowing of focus, at the first-year undergraduate level, the initiative might do well to 
encourage students to broaden their considerations of a variety of course and career options. 

Awareness of discipline. It was hypothesized that through the research experience 
students would gain an increased understanding of the discipline that was the focus of their 
course, and of the research process in that discipline. One teaching strategy intended to increase 
disciplinary awareness was to expose students to tools and methodologies of the discipline. For 
social sciences, this might be survey or interview methodology, while for agriculture or 
environmental science, students this might be taking soil samples. This strategy was expected to 
increase awareness of the discipline by allowing students to experience the types of work that a 
researcher in their field engages in on a regular basis. Although responses covered the full range 
of available options, undergraduate students, on average, marginally agreed that they had a 
greater understanding of their discipline a result of the research experience (see Figure 8). 



Strongly Disagree Neither Agree Strongly 
disagree agree nor agree 

disagree 


■ Students (M = 3.28) 


Figure 8. Increased awareness of discipline. 


In term of research process, instructors substantially agreed that the participating students gained 
an increased understanding as a result of the course-based research experience. Research coaches 
also agreed with this statement, but with more variation in responses. Undergraduate students’ 
responses were similar to those of the research coaches, demonstrating more modest agreement 
with this outcome statement (see Figure 9). 
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Strongly Disagree Neither Agree Strongly 
disagree agree nor agree 

disagree 


■ Students (M = 3.57) 

□ Research Coaches (M = 3.67) 

□ Instructors (M = 4.40) 


Figure 9. Increased understanding of research process. 


In the focus groups and interviews, some instructors and research coaches expressed 
certainty that the students gained an understanding of the research process in their discipline. 
These assertions were usually tied to specific examples of how the research process was carried 
out in their labs or the class proper. They suggested that it would virtually be impossible for 
students to have not gained this knowledge, as it was the entire focus of the experience. 

Other faculty suggested that rather than a firm grasp of exactly what research entails, the 
students may have gained an understanding of the potential research holds. They described how 
they did not want students to finish their class believing they were experts in research, but that 
the students would leave understanding that they had just touched the surface of what research is 
and what it can accomplish. 

Finally, many believed the experience was a pleasant introduction into what can often be 
a daunting area. They suggested that this was an interesting, engaging, and fun experience for the 
students that contrasts with the typically anxiety-provoking reputation of research-heavy courses. 
They also described how participating in the research experience would make students not only 
more informed about conducting research, but also informed participants and consumers of 
research. 

Skills development. It was anticipated that students’ professional skills (e.g., working in 
groups, critical thinking, presentations skills, problem solving) would improve as a result of 
participating in the research experience. The expectation was that since students would have to 
exercise these skills throughout the experience, these skills would begin to develop or progress. 
Students, on average, slightly agreed that their professional skills had improved as a result of the 
initiative. Research coaches’ perceptions of students’ professional skills improvement was 
similar to that of the students, while instructors’ perceptions expressed greater confidence (see 
Figure 10). 
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Strongly Disagree Neither Agree Strongly 
disagree agree nor agree 

disagree 


■ Students (M = 3.41) 

□ Research Coaches (M = 3.67) 

□ Instructors (M = 4.60) 


Figure 10. Improved professional skills. 


In the interviews and focus groups, some research coaches and instructors suggested that it was 
the experience with presentations that enhanced students’ professional skills. Some viewed the 
main benefit for undergraduate students as the improvement in practical research skills such as 
conducting literature reviews, writing, and understanding the sources of and process of research. 
Finally, the instructors and research coaches reported that gains in students’ professional skills 
were largely related to learning to work in a group. What processes occurred in the group work 
for the undergraduate students, and how, in turn, the results lead to improved professional skills 
were explained. One faculty member described the course-based research experience as a fairly 
unique opportunity for students to gain these types of professional skills: 


[they learned] problem solving, team work, following instructions, interacting with 
people, absolutely! I think all of that just feeds into professional skills that, quite frankly, 
I don’t think all first year students gain if they don’t become engaged in these types of 
class experiences. 

Student engagement. In part, the staff and university administration launched the UGR 
initiative in course-based undergraduate research experiences for first-year students with the goal 
of increasing student engagement in learning. On the whole in the surveys, students marginally 
agreed that the research experience caused them to feel more engaged in learning about the 
class’s subject matter. Research coaches’ survey responses demonstrated more overall agreement 
that first-year students were more engaged in learning as a result of the research experience, 
while instructors’ agreement was even more consistent and enthusiastic (see Figure 11). 
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Strongly Disagree Neither Agree Strongly 
disagree agree nor agree 

disagree 


■ Students (M = 3.23) 

□ Research Coaches (M = 3.60) 

□ Instructors (M = 4.0) 


Figure 11. Increased engagement in learning. 


In the focus groups and interviews, research coaches and faculty suggested that a variety of 
factors explained why the students may have been more engaged as a result of the initiative. In a 
number of classes, the research data were collected from the class members themselves, and 
some instructors suggested that this led the students to be more interested in class than they 
might have otherwise been. Instructors believed the students were excited to learn about what 
their peers thought about certain issues, and that the anonymous nature of the surveys allowed 
for more honest responses than would have been accomplished with a class discussion. 

Some classes participated in a poster session at the end of the term. A number of 
professors suggested that the research project and corresponding poster presentation resulted in 
more engagement and pride than would have resulted from a standard assignment, such as a term 
paper. One instructor specifically mentioned, “I think they became very proud of their poster and 
their topic. Whereas in previous years, I don’t think they really felt any connection to any term 
paper.” 

Some instructors suggested that the freedom to choose their own research topic was key 
to engagement, and that this led students to be more excited than if they had completed an 
assignment on a pre-set topic. One instructor mentioned that levels of engagement were 
maintained with the research project, because it added variety to their pedagogical practices with 
the class. 

Identification as student-researchers. Another goal of the research experience was for 
students to have an increased understanding of how researchers think and work, and in turn to 
begin thinking and working like researchers. As a group, the students moderately agreed in their 
surveys that they had a better idea of how researchers think and work as a result of their research 
experience (see Figure 12). 
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Strongly Disagree Neither Agree Strongly 
disagree agree nor agree 

disagree 


□ Students (M = 3.44) 


Figure 12. Better idea of how researchers think and work. 


Although students believed they gained a slightly greater understanding of how researchers think 
and work, they were marginally less likely to agree to feeling they could personally think and 
work more like researchers since the research experience (see Figure 13). 



Strongly Disagree Neither Agree Strongly 
disagree agree nor agree 

disagree 


■ Students (M = 2.99) 


Figure 13. Personally think and work more like researcher. 


Research coaches and instructors however, subscribed to the belief that the students had begun to 
understand they have a role in research on campus (see Figure 14). 
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Strongly Disagree Neither Agree Strongly 
disagree agree nor agree 

disagree 


Figure 14. Role in research on campus. 


0 Research Coaches (M = 3.80) 


□ Instructors (M = 4.0) 


In the focus groups and interviews, instructors and research coaches were reluctant to 
wholeheartedly agree that undergraduate students were beginning to identify as researchers. 
However, coaches and faculty suggested that the experience provided the students with a sense 
of their potential to be researchers, or at least a sense of agency in terms of their research 
interests and future research activities. 


Research Coaches 

Research coaches were upper level undergraduate students and graduate students. 
Accordingly, the extent to which they had prior experiences similar to research coaching (e.g., 
teaching, assisting with teaching, marking) varied substantially form research coach to research 
coach. Thus the degree to which this was a unique skill building experience for individuals may 
have varied by research coach. Overall, research coaches indicated in the surveys that they felt 
that they benefited through their experience as a research coach (M = 4.29, SO = .61). They also 
agreed that they benefitted from working closely with a faculty member or team of research 
coaches (M = 4.0, SD = 1.13). In particular, research coaches indicated that they felt the 
experience helped them develop their professional skills (i.e., critical thinking, problem solving, 
interpersonal skills). Instructors also believed the research coaches’ professional skills improved 
as a result of their role (see Figure 15). 
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disagree agree nor agree 

disagree 


El Research Coaches (M = 4.13) 
□ Instructors (M = 4.20) 


Figure 15. Improved professional skills. 


On average in the surveys, research coaches also agreed that they learned new teaching methods 
during their time fulfilling this role (M = 3.6, SD = 1.40), and that they see research and teaching 
as more connected as a result of the initiative (M = 3.80, SD = 1.08). For both these items, 
however, there was a full range of responses (range 1.0 - 5.0) indicating some research coaches 
strongly disagreed with these statements while other strongly agreed. 

In the focus groups and interviews, research coaches indicated that their professional 
skills improved in number of ways from participating in the initiative. One research coach said: 

[I]t goes back to cv building and building those skills, because if you’re planning on 
doing any sort of graduate degree, it doesn't matter what the discipline is, chances are 
these are things you’re going to have to do...If you’re doing your PhD you’re probably 
going to be responsible for or teaching a class. So learning how to give good critical 
feedback, deal with students, things like that. I think it is super beneficial! 


Some research coaches suggested that the experience in lesson planning was a benefit to them. 
Many mentioned that experience leading research instruction and mentoring was valuable for 
building their curriculum vitae. They suggested that learning how to give critical feedback and 
explain difficult concept to students was a primary benefit. One coach noted: 


[BJecoming comfortable in front of a classroom, more comfortable working with 
students, getting a better understanding of what research entails myself... and how to 
explain it to the students, that was definitely a big one [benefit], was how to explain 
things to students that are not understanding. 


A more senior student mentioned that working with the research experience was simply another 
opportunity to build mentoring skills, but still a valuable one. Many research coaches mentioned 
that the experience with facilitation and leading discussions was a great opportunity that allowed 
them to become more comfortable in preparation of teaching roles. Finally, one research coach 
explicitly mentioned benefitting from being exposed to the idea of a research coach in contrast to 
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their previous conceptions around being a teaching assistant. They reported that the slight shift in 
ideology between the two roles was welcome. One coach said: 

[I]t is a little bit of a better way, a more interactive way to think about what you’re trying 
to teach them.. .trying to get away from this ‘guy behind a pulpit’ [role], just putting stuff 
on the board so they can record it... The change in ideology, the change in mindset was 
kind of nice. 

Instructors echoed the belief that the experience was beneficial for the research coaches 
in terms of their professional skills. Many instructors agreed that the experience in front of a 
class is invaluable. Further, they suggested that depending on the research coach, giving them 
autonomy was also a beneficial experience because it engendered critical thinking about how to 
lead a class. 

Faculty Instructors 

More than either of the other participant groups, faculty strongly agreed that they 
benefitted from their time with the initiative (M = 4.60, SD = .52, range 4.0 - 5.0). In their survey 
responses, faculty agreed strongly that they learned new teaching methods and approaches 
during their time with the initiative (M = 4.4, SD = .7, range 3.0 - 5.0), that because of the 
initiative they see teaching and research as more connected (M = 3.70, SD = 1.16, range 2.0 - 
5.0), and that because of participating they are more invigorated about research (M = 3.90, SD = 
.79, range 3.0 - 5.0). Instructors described a number of ways that participating in the initiative 
benefitted them. For some, the benefits were more for the class than for themselves individually. 

The benefits instructors reported were often related to the initiative inviting them to think 
more about why they are choosing certain pedagogical activities for certain courses. A few 
instructors reported that implementing a research experience for their students made them more 
interested in pedagogical engagement strategies generally and for non-traditional learners 
specifically. For instance, one instructor said, “I think it’s increased my interest in different ways 
to engage students in the classroom, especially students that are less comfortable with an 
academic tradition.” 

Some faculty noted that they benefitted from a refreshed mindset as a result of the UGR 
initiative, and that the experience made them rethink and question their assumptions about 
teaching this class. One faculty member specifically stated, “It made me reflect on the way I 
teach, the way I could teach, different modalities, different methodologies, and different 
strategies.” 

Program Implementation 

Overall, instructors were quite satisfied with how the research experience was 
implemented (M = 4.1, SD = .57), and this was rather consistent across instructors (range 3.0 - 
5.0). Research coaches (M = 3.6, SD = 1.18) and students (M = 3.6, SD = 1.05) were also 
generally satisfied with how the research experience was implemented in class. Some common 
themes in classes that appeared to demonstrate the most success with outcomes and satisfaction 
with process included: 
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• A collaborative and supportive relationship between the instructor and research coach or 
group of research coaches. This appeared to be beneficial for a number of reasons. First, 
research coaches who described a collaborative relationship with their instructor 
reported more personal and professional benefits of participating as a research coach. 
These were the coaches that participated in lesson planning, delivering research content, 
facilitating, and developing the research experience. Second, this type of relationship 
was perceived as beneficial because these research coaches felt they had a better 
understanding of their role, responsibilities, and the means to accomplish the goals of 
their work. 

• Consistent attendance and participation in the UGR events, meetings, and workshops. 
These instructors had a better picture of the goals and processes of the course-based 
undergraduate research initiative. They perceived themselves, their research coaches, 
and the UGR staff as a team working towards the same goals. Faculty in this group were 
more invested in the project and had a clearer understanding of what they were working 
towards. 

• Using tools and methodologies of the discipline. If the purpose of the research 
experience is to provide a greater understanding of research in a field, using the tools 
and methodologies of that discipline would be advisable. It can be difficult to use 
discipline-relevant tools and methodologies in the social sciences because of ethical 
issues in working with human participants (i.e., most research with human participants 
requires ethical clearance, which is difficult and time consuming for an introductory 
class of 300) and difficult in the natural sciences because of the expense of natural 
sciences research (e.g., much research in agriculture or bioscience requires particular 
tools or samples that are expensive to provide to an introductory class of 300). Small 
scale or shared projects may be solutions for these issues. 

• Laboratory visits and expert guest speakers. Students in classes containing these 
elements seemed to gain the greatest understanding of how research works and the goals 
and importance of research on campus. These learning strategies were not necessarily 
easy to organize and the UGR initiative and other campus resources may be able to 
provide support in organizing or facilitating these trips and speakers. 

• Topics and projects that integrate, support, or apply course content. This was noted as 
difficult to integrate at times, but highly beneficial. These elements were seen as helpful 
to students’ assimilating and accommodating new knowledge and facilitative for 
students’ learning both content and methodology. 

Discussion 

Promise 

The results of this evaluation suggest that first-year CUREs show promise in terms of 
benefiting undergraduate student participants as well as the faculty and research coaches 
involved. Faculty were the participant group that reported the highest levels of benefitting from 
the initiative for themselves and the other participant groups. The primary benefits that faculty 
reported experiencing included an increased interest in ways to engage learners, reexamination 
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of their teaching strategies, the pragmatic support of a research coach helping with their work 
load, and an invigoration of their research. The primary benefits to research coaches included 
increased professional skills via experience in lesson planning and delivery, gaining facilitation 
skills, CV building and an ideology shift in how to best facilitate learning for undergraduate 
students. 

Of all the outcomes considered for undergraduate students, the most prominent benefits 
appeared to be that they gained a better idea about how researchers think and work, that they 
increased their understanding of how research works, and that their own research and 
professional skills had improved. From the perspectives of the faculty and research coaches, the 
program was a pleasant introduction into what can be a daunting field for new undergraduates. 
Faculty and coaches perceived the experience to have helped eliminate methodology “phobia” 
among first-year students and decreased their traditional hesitancy to take methods courses. 
Further, faculty and coaches believed students gained an awareness of careers options and 
possibilities, suggesting also that students had become more informed research consumers and 
participants as a result of the in-class research experience. Faculty and research coaches 
described how they saw undergraduate students’ skills in group-work, presentations, research, 
and academics improve. Students were perceived to have exhibited more pride in their research 
projects than they typically do in more traditional assignments. 

Classes were not uniform in terms of their approach, reported achievement of outcomes, 
or satisfaction with the process. Likewise, research coaches exhibited a range of satisfaction with 
process and perceived outcome achievement for themselves. Aspects of implementation that 
seem to best facilitate achievement of the initiative’s expected outcomes included: a 
collaborative and supportive relationship between the instructor and research coach(es); 
consistent instructor attendance and participation in the UGR events, meetings, and workshops; 
using discipline-relevant tools and methodologies; engaging in laboratory visits and inviting 
expert guest speakers; and focusing on topics and projects that integrate, support, or apply course 
content. 

Pitfalls and Potentials 

Some outcomes from the students’ perspectives that did not appear to be achieved as 
often or as much included increased clarity about what field they would like to pursue, whether 
they had an interest in graduate studies, increased engagement in class, and an increased sense 
that they had developed to think and work like researchers. Although faculty agreed that students 
were not quite identifying as researchers yet, faculty and coaches believed that students did gain 
a sense of their potential to become researchers. Focus group discussions mirrored the students’ 
survey responses in that faculty and research coaches did not believe students gained concrete 
ideas about their future. Faculty also suggested that perhaps the particular goals of clarity around 
career interests or beginning to think and work like a scientist may be inappropriate for a first- 
year research experience. 

These outcomes were anticipated for the UGR initiative because previous research 
(Hunter et al., 2007; Lopatto, 2009; Seymour et al., 2004) has indicated that benefits to students’ 
participating in UREs include thinking and working like a researcher, becoming a scientist, 
clarification, confirmation, and refinement of career/education paths, and enhanced career and 
graduate school preparation. However, most previous research and evaluation of UREs generally 
has focused on upper year science students, most often completing a summer apprenticeship 
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where they work one on one with a faculty or graduate student mentor. It may be more likely that 
these senior students, being farther along in their education, and already more specialized, are at 
a stage where they are prepared for, and acknowledge, these academic and professional gains. 
Also, summer student research apprentices are often either self-selected or invited by a faculty 
member to engage in research. Rather than being an optional opportunity for particularly 
promising students, CUREs involve all students in the participating first-year classes. 

Thus, it may be a pitfall, and potentially setting the initiative up for failure to expect that 
students in first-year CUREs will experience identification as a researcher, or clarity of their 
careers goals. When considering these outcomes, faculty participants suggested gaining 
awareness and understanding their potential rather than experiencing clarity might be more 
appropriate and achievable goals of CUREs with first-year students. They suggested students 
gained awareness of career/field or options or possibilities, awareness of the potential of 
research, awareness of their own potential to be researchers, respect for and exposure to multiple 
fields and ideas, and self-confidence approaching methodology. Thus, the UGR and similar 
research initiatives at other universities may consider changing their emphasis from outcomes 
that involve narrowing first-year students’ focus to outcomes that involve broadening their 
horizons. Similarly, given the main outcomes experienced by the faculty/instructors focused on 
an increase in self-reflective teaching practice, the UGR and similar initiatives might consider 
changing their outcome focus from increased use of new or alternative pedagogical practices to 
facilitating self-reflective teaching and research practice (which might very well include new or 
alternative practices) for faculty and research coaches. 

Limitations of the Evaluation 

Some limitations warrant discussion. First, the evaluation was unable to secure 
qualitative data from undergraduate student participants. This data could help understand the 
students’ perceptions of the research experience, the extent to which outcomes are being 
achieved, and the discrepancy between students’ and faculty’s assessment of the extent to which 
student outcomes are being achieved. Second, the evaluation entailed a cross-sectional 
descriptive design, and thus the evaluation cannot speak to the causal role of the initiative in the 
achievement of outcomes, persistence of initial outcomes, or achievement of long-term 
outcomes. We can only report that at this stage of the pilot, these are the participants’ perceptions 
of the initiative’s effects. Relatedly, third, all data was self-reported, based on individuals’ 
perceptions and impressions. Although this is appropriate to understand issues such as 
instructors’ invigoration in research or undergraduates’ clarity about course selection, outcomes 
such as improved academic performance could be measured with more objective methods. 
Fourth, although all survey items and focus group questions were focused on the research 
experience proper, it is difficult to know the extent to which general opinions about the classes 
(e.g., students’ thoughts about the instructor or course content; instructors’ attitudes towards the 
cohort of students) affected responses. Fifth, there was a lack of standardization of mode of 
survey delivery (i.e., in person, pen and paper versus online) and format of qualitative data 
collection (i.e., interview versus focus groups). Although this may not be ideal, it was necessary 
to accommodate the real-world constraints of our participants. In addition, although modes and 
formats were not standardized, items and questions were, and, similar themes emerged from the 
interviews and focus groups, suggesting that the format of data collection did not noticeably 
affect the responses. 
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Finally, in all cases, faculty/instructors were more certain than research coaches and 
students that the students experienced the expected benefits of the program. There are a number 
of explanations that could underlie this finding. First, faculty may have perceived themselves as 
owners of the project and may be more personally invested in the success of the program, and 
thus biased in their assessment. On the other hand, faculty may simply have a different vantage 
point for comparison. That is, they have had the opportunity to observe students across a number 
of years and classes. They have seen the levels engagement, clarity, and skill of umpteen 
students in other classes who did not have the research experience. Thus they may have ta 
different baseline with which to compare and formulate opinions about these students’ 
experience. The undergraduate students might not perceive the benefits to them, because they 
might not be aware of a potentially different situation or the full-range of benefits. To them, the 
research experience may be the standard to which they have little to compare. 

Interestingly, faculty was also the participant group that reported the highest levels of 
agreement with the personal proposed benefits as a result of the initiative. As above, they may 
have actually benefitted the most, or have had years of teaching and research practice with which 
to compare their recent experience, and thus are able to notice the changes from this point of 
comparison. Regardless, faculty appear to be highly satisfied with the outcomes of the 
experience. 


Conclusion 

The first-year undergraduate research experience at the University of Saskatchewan 
shows great promise to benefit the participating students, research coaches and faculty. 
According to this preliminary evaluation, first-year research experiences may benefit faculty by 
encouraging self-reflective teaching practices, they may benefit research coaches by preparing 
them for future teaching and facilitation roles, and they may benefit undergraduate students by 
helping them improve their research and professional skills and gain a better understanding 
research. Early, bottom-up evaluation identified characteristics of implementation that appear to 
best facilitate achievement of the initiative’s outcomes and identified the potential pitfall of 
imposing outcomes, from related but distinct initiatives, that may not be achievable or optimal in 
in the setting of first-year classes. The results of this evaluation suggest that rather than gaining 
clarity or focus, first-year students in CUREs might gain awareness of their personal potential, of 
the potential of research, and of their career/educational options. 
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Appendix A 
Student Survey Items 

1. I thought the research experience was adequately organized. 

2. I understood the purpose of the research experience when it was introduced. 

3. I think I have an understanding of research: what it is and how it works. 

4. In the research we conducted, I contributed to creating a research question. 

5. To my knowledge, we investigated our research question. 

6. We shared or disseminated our results with someone other than our instructor. 

7. Because of the research experience, I have a better idea about what courses I would like 
to take (or not to take) in the future. 

8. Because of the research experience, I have a better idea about what subject I want to 
major in (or what subject I know I do not want to major in). 

9. I think I have an increased understanding of [insert discipline] because of the research 
experience. 

10.1 think I have an increased understanding of how research works because of the research 
experience. 

11.1 believe my research skills have improved as a result of the research experience. 

12.1 believe my professional skills (i.e., critical thinking, problem-solving, and getting along 
in groups) have improved as a result of the research experience. 

13. I believe my academic skills (i.e., finding and reading journal articles, summarizing 
research findings) have improved as a result of the research experience. 

14.1 found the research experience to be engaging (interesting and captivating). 

15. The research experience made me more engaged (interested) in class. 

16. The research experience made me more engaged in learning about [insert discipline]. 

17.1 have a better idea about what I would like to pursue (or not pursue) as a career because 
of the research experience. 

18. Because of the research experience, I have a better idea about my interest (or lack 
thereof) in graduate studies (i.e., pursuing a master’s degree or PhD). 

19.1 think I have a better idea about how researchers think and work as a result of the 
research experience. 

20.1 feel like I think and work more like a researcher since the research experience. 

21.1 believe participating in the research experience will contribute to my academic success. 

22.1 think I have an understanding of the purpose of research on campus. 

23.1 have discussed the good things about my research experience with other people on or 
off campus. 
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Appendix B 

Research Coach Survey Items 

1.1 felt satisfied with the research coach training/training experience. 

2.1 felt the training experience was well organized. 

3. Training provided me a clear understanding of the role of the UGR initiative on campus. 

4. Training provided me a clear understanding of my role as research coach. 

5.1 benefited through my experience as a research coach (beyond being paid). 

6. The research coach experience helped me develop my professional skills (i.e., critical 
thinking, problem solving, interpersonal skills) 

7.1 plan to work on a scholarship of teaching and learning (SoTL) publication. 

8.1 learned new teaching methods during my time as a research coach. 

9.1 plan to use these methods in my future group facilitation or teaching. 

10.1 see research and teaching as more connected as a result of participating as a research coach. 

11.1 feel like I benefitted from working closely with a faculty member or team of research 
coaches and tutorial leaders. 

12.1 think participating as a research coach will contribute to my own academic success. 

13. In class, I think the ‘research arc’ was adequately explained to the undergraduate students. 

14. To my knowledge, the class spontaneously developed a research question (it was not the 
instructors question). 

15. The results of the research experience were disseminated outside of class. 

16. Overall, I was satisfied with how the UGR experience was implemented in class. 

17.1 think undergraduate students developed a good understanding of research on campus. 

18.1 think the undergraduate students developed a good understanding of the goals and 
importance of research at the University of Saskatchewan. 

19.1 think undergraduate students began to understand they have a role in research on campus 

20.1 think the undergraduate students have an increased understanding of the research process as 
a result of the in-class research experience. 

21.1 believe the research experience has contributed to improving the undergraduate students’ 
professional skills. 

22.1 believe the undergraduate students are more engaged in learning as a result of the in-class 
research experience. 

23.1 think the research experience provided the undergraduate students increased clarity 
regarding what courses they would like to take in future. 

24.1 think the research experience provided the undergraduate students increased clarity 
regarding what field they might like to pursue at university. 

25.1 think the research experience provided the undergraduate students increased clarity 
regarding if they may be interested in pursuing graduate studies. 

26.1 think the research experience provided the undergraduate students increased clarity 
regarding their career interests. 
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Appendix C 

Instructor Survey Items 

1.1 felt satisfied with the preparatory discussions and support the UGR Initiative team provided 
me. 

2.1 felt the experience was organized 

3. The Initiative team supported me in developing an understanding of the role of the UGR 

initiative on campus and its goals. 

4. The initial discussions allowed me to develop a clear understanding of the role of the 

instructor in the UGR initiative on campus. 

5.1 benefited through my experience with the Initiative. 

6.1 learned new teaching methods and approaches. 

7.1 applied new teaching methods and approaches. 

8.1 plan to use these methods/approaches in my future teaching. 

9.1 see research and teaching as more connected as a result of this experience. 

10.1 plan to integrate teaching and research more in the future. 

11.1 feel more invigorated about research as a result of the experience. 

12. I’m interested in conducting more research as a result of the initiative. 

13. I’m interested in conducting more types of research as a result of the initiative. 

14.1 have worked (or plan to work on a Scholarship of Teaching and Learning (SoTL) 
publication. 

15.1 used the concept of the “research arc” or the process of research in my discipline in class 

16. My students created their own research questions. 

17. Students investigated their questions using discipline-relevant tools and methodologies. 

18. Overall, I was satisfied with how the UGR experience was implemented in class. 

19.1 think the undergraduate students developed an initial understanding of research on campus. 

20.1 think the undergraduate students developed an understanding of the goals and importance 
of research at the University of Saskatchewan. 

21.1 think the undergraduate students began to understand they have a role in research on 
campus. 

22.1 think the undergraduate students have an increased understanding of the research process as 
a result of the in-class research experience. 

23.1 believe the research experience has contributed to improving the undergraduate students’ 
professional skills (e.g., critical thinking; group work). 

24.1 believe the undergraduate students are more engaged in their own learning as a result of the 
in-class research experience. 

25.1 think the research experience provided the undergraduate students increased clarity 
regarding what courses they would like to take in future. 

26.1 think the research experience provided the undergraduate students increased clarity 
regarding what field they might like to pursue at university. 

27.1 think the research experience provided the undergraduate students increased clarity 
regarding if they may be interested in pursuing graduate studies. 

28.1 think the research experience provided the undergraduate students increased clarity 
regarding their career interests. 

29.1 believe the research coaches’ professional skills (e.g., facilitation, leadership) have 
improved as a result of participating as a research coach. 
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30.1 believe the research coaches’ research skills have improved as a result of their participation. 

31.1 believe the research coaches will have increased employability in academic or related fields 
as a result of participating as a research coach. 

32.1 found the support of a research coach to be a valuable form of support for me in 
implementing the research experience. 
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Appendix D 

Research Coach Interview/Focus Group Questions 

1. We’ll start with some questions about your training. What did your training entail? 

• What type of training did you receive training in professional conduct? 

• Program goals? 

• Facilitation? 

• Teamwork? 

• Interpersonal skills? 

2. Were you satisfied with your training/training experience? 

• Did you feel like it was well organized? 

• What could have been improved? 

• What was helpful/useful? 

• Was anything redundant or unnecessary? 

3. After training, did you feel had a clear understanding of the role of the UGR initiative on 
campus? 

• What is that role? 

• What are the goals of the initiative? 

• Did your training inform you of these goals? 

4. After training, did you feel you had a clear understanding of you role as research coach? 

• How would you describe that role? 

5. Now I’m going to ask some questions about the benefits to you of participating in the 
initiative. Do you feel there were benefits to you (beyond being paid)? 

6. Do you feel like your research or professional skills have improved as a result of 
participating in the initiative? 

• How so? Describe. 

7. Did any of you work on any scholarship of teaching and learning (SoTL) work during your 
time as research coach? 

• Are you aware of SoTL? 

• Do you plan to engage in SoTL? 

• How could you be supported in this? 

8. Did you learn about any new or interesting teaching methods during your time with the 
initiative? 

• Might have also been called “alternative pedagogical practices” 

• Do you plan to use some of these methods or practices in your own teaching? 

9. Do you think about research and teaching any differently as a result of the initiative? 

• Do you see them as any more connected? 

• What are barriers to this idea? 

• Do you think the initiative challenged the traditional teaching/research dichotomy? 

• Do you think this has changed how you will approach research and teaching in your 
future practice? 

10. In what ways do you think that participating in the initiative might contribute 1 to your own 
academic success? 
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11. Now I’m going to ask you some questions about how the UGR experience was actually 
implemented in class. 

• Do you believe that the “research arc” was adequately explained? 

• Did the class develop a research question spontaneously (i.e., not the instructors 
question)? 

• Was the research question investigated according to the tools and methodologies of 
your discipline? 

• Were the results disseminated? How? 

12. Overall, were you satisfied with how the UGR experience was implemented in class? 

• Did you feel like it was well organized? 

• What could have been improved? 

• What was well done? 

• Was anything redundant or unnecessary, or poorly executed? 

13. Now I have a few questions about the UG students’ thoughts about the UGR initiative: 

• Do you think the UG students had a clear understanding of the role of the UGR 
initiative on campus? 

• Do you think the UG students have a clear understanding of the goals of the initiative? 

• Do you think the UG students have a clear understanding or their role in the initiative? 

14. Finally, a few questions about the potential benefits of the initiative for undergraduate 
students who participated: 

• Do you think the UG students have an increased understanding of the research process? 

• Do you think the UG students’ research and professional skills have improved? 

• Do you think the UG students’ are any more engaged in learning as a result of the 
initiative? 

• Do you think the UG students have increased clarity regarding their career interests and 
interest in graduate studies due to the initiative? 
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Appendix E 

Instructor Interview and Focus Group Guide 

1. We’ll start with some questions about the discussions and supports the UGR provided you. 
What did your preparation for participation entail (meetings, workshops; face-to-face events; 
individual consultations)? 

• Did you access and/or utilize to new teaching strategies, approaches, or assessment 
considerations? 

• UGR Program strategies and goals? 

• Cooperative learning? 

2. Were you satisfied with the preparatory process? 

• Did you feel like it was organized? 

• What could have been improved? 

• What was helpful/useful? 

• Was anything redundant or unnecessary? 

3. After preparation, what was your understanding of the role of the UGR initiative on campus? 

• What is your understanding of the goals of the initiative? 

• After initial discussions, did you feel you had a clear understanding of what your role 
could be in the undergraduate research initiative? 

4. Now I’m going to ask some questions about the benefits to you of participating in the 
initiative. Do you feel there were benefits to you, your department, or your college 
(professionally, pedagogically, and research-wise)? 

5. In what ways has the experience enhanced your knowledge of additional pedagogical 
practices? 

• Describe. Or give an example. 

• Do you plan to apply some of these practices in your future teaching? 

6. Do you think about research and teaching any differently as a result of this experience? 

• How? Why or why not? 

• Do you see them as any more connected? 

• What are barriers to this idea? 

• Do you think the in-class research experience challenged the view that 
teaching/research are a dichotomy? 

• Do you think this experience has changed how you will approach research and 
teaching in your future? 

• Do you plan to integrate research and teaching more in the future? 

7. In what ways has the experience affected how you approach your own research? 

• Are you any more or less excited, invigorated or enthusiastic? 

8. Has the experience affected the range of research you conduct or are interested in? 

• Different types of methodologies perhaps? 

• New topic areas? 

• What about the types of research you value? 
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9. Have any of you been working on any Scholarship of Teaching and Learning (SoTL) work 
(or plan to)? 

• Are you aware of SoTL? 

• Do you plan to engage in SoTL? 

• How could you be supported in this? 

10. Might you involve undergraduate students any differently (more or less) in your own 
research as a result of this experience? 

11. I’m going to ask you some questions about how the UGR experience was actually 
implemented in class. 

• Did you use the concept of the “research arc” (or process of research according to your 
discipline)? 

• Did students develop their own research questions? 

• Were the research questions investigated (according to relevant tools and 
methodologies of the discipline)? Why or why not? 

• Were the results disseminated/shared? How? (How well did this work?) 

12. Now I have a few questions about the UG students’ thoughts about research on campus. 

• Do you think the UG students gained an understanding of research? 

• Do you think the UG students gained an understanding of the goals and importance of 
research at the University of Saskatchewan? 

• Do you think the UG students gained an understanding of their role in research on 
campus? 

13. Now, a few questions about the potential benefits of the first-year in-class research 
experience for undergraduate students who participated: 

• Do you think the UG students have an increased understanding of the research process? 

• Do you think the UG students’ research and professional skills (e.g., critical thinking, 
working in groups, etc...) have improved? 

• Do you think the UG students’ are any more engaged in their own learning as a result of 
the initiative? 

• Do you think the UG students have increased clarity regarding their career interests, 
what courses to take, what field might interest them, and/or will show an increased 
interest in graduate studies due to the in-class research experience? 

14. Finally, a few questions about the research coaches: 

• Did you find the salary support of a research coach to be a valuable form of support for 
you in implementing the research experience? Is it your preferred form of support? 

• How has the experience affected their research or professional skills? (Improved?) 

• Do you see them having increased employability in academic or related fields? 
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